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ABSTRACT 

Two years ago the University of Tampa acquired the 
"BASIC SKILLS" software program developed by Computer Systems 
Research to help underprepared students bring their reading, writing, 
and mathematical skills up to college levels* Two studies were 
undertaken to determine the effectiveness of the software and to 
establish the most effective methods for using it. One study focused 
on regular freshman composition classes, while the other sampled 
developmental writing courses. The results most favorable for the use 
of the program came from the research into freshman composition 
groups. These results suggest that using the program regularly may 
help to improve student performance. Students working with the 
program had fewer sentence level mistakes and said that they enjoyed 
using the program. The results from the research focused on 
developmental writers showed surprising trends. Students using the 
program actually tended to increase the number of errors over time. 
Factors regarding the small number of sampled students may have 
overly influenced these findings. Student motivation increases as the 
use of the computer program is more directed tov^^rd individual 
student weaknesses. But it is difficult to explain the consistent 
increase of errors as developmental students used the program 
regularly. Overall, findings caution against overgeneral izat i on 
regarding the effectiveness of the "BASIC SKILLS" program. 
Additionally, regular composition students seem more suited to the 
program than basic writers. (HB) 
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Abstract: Two years ago the University of Tampa acquired the BASIC 
SKILLS software program developed by Computer Systems Research 
(CSR) to help underprepared students bring their reading, writing, and 
mathematical skills up to college level. In an attempt to determine the 
effectiveness of the software and to fmd the most effective way of using it, 
we undertook two studies of its use: one with regular freshman 
composition classes and one with developmental writing classes. This 
article summarizes the results of our studies, discusses student and faculty 
responses to the program, and offers recommendations for further use and 
study of BASIC SKILLS as a writing tutor. 



U.S. OCMirfMENT or f OUCATKM 
Ottpcf Educaionat R*MaiCh and tniprov9m«nt 
EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



Thit document hat b««n '•p/txluccKj %t 
(«c*<ved from the p#fion Of Ofg»nu«t»on 

Ongifvtttng it 




• P<Nntl Ol ytmw or opinion* stst«<) m this docw 
m«nt do not n«c«SMrily r«pr«Mnt otiKM 
0€HI pociton or po(K> 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) - 



2 



BEST copy AViUUBLE 



Computers as Writing Tutors 



Laura enters the computer lab, types in "CSR" and her 3-digit user 
code, and brings up the grammar module on sentence fragments. She takes 
a ten-question pretest on fragments, and her score of 80% indicates that she 
does not need further instruction in the skill. Will her successful 
performance on the computer grammar module transfer to her writing? 
That*s the question our study of the BASIC SKILLS program of CSR 
(Computer Systems Research) attempted to answer. 

When our university, the University of Tampa, decided cn a 
computer program for remediation and enrichment in writing, reading, 
and math written for grade levels 8, 11, and 13, many of our colleagues 
were skeptical about how well the program could deliver. 

"We can't rely on a machine to solve our students* problems,*' one 
professor stated, while another reminded us that grammar could not be 
taught in isolation. Some math department members said they simply didn*t 



feel the computer modules would make a significant difference in their 
students'learning. 

We knew that Miami-Dade Community College, with a population of 
28,000 students, was conducting its own computer efficacy study of CSR 
and similar computer-assisted instructional programs such as Realtime and 
PLATO. Project Synergy, as the Miami-Dade study is called, describes 
classroom research of nine faculty who worked with their students in an 
Electronic Classroom, Comparison studies of control and experimental 
groups gave generally favorable reviews to the effects of computer-assisted 
instruction, especially with underprepared students {Project Synergy,\99l). 
Students especially liked the one-on-one interaction with the computer, the 
game-like quality of some of the exercises, and the opportunity to do both 
tutorials and pretests. 

The purpose of our study, however, is not to make direct statistical 
comparisons between our results in the use of CSR and Miami-Dade s 
results. The two institutions are too different to make convincing 
comparisons . For example, students at community colleges such as 
Miami-Dade tend to drop classes far more readily than students at four- 
year institutions such as the University of Tampa. Such a difference could 
easily affect the pass rate in classes under study and so render comparison 
between our statistics and Miami-Dade's invalid. 

Our study of CSR use is designed as descriptive inquiry. We want to 
describe our students' actual performance with and without the help of 
CSR, as well as the students' experiences with and responses to the 
software. Such description in itself should be of interest and use to anyone 
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considering computer-assisted instruction in basic composition courses. 
Furthermore, we want to use this description as a basis for inquiry into the 
reasons for students' apparent success or lack of success with CSR. The 
principal questions we wanted to answer were these: 

(1) Would working with CSR make a noticeable difference in 

the performance of the students in our experimental group? 

(2) How would the performance and improvement of students in the 

experimental group compare with the performance and 
improvement of students in the control group / 
These questions have been answered, though the answers are 
different for the regular composition classes and the developmental classes. 
But our answers have raised other questions which invite further 
investigation into the efficacy of the CSR program. 

CSR'S BASIC SKILLS PROGRAM 

The BASIC SKILLS program is designed to help students from 
elementary school through college acquire necessary skills in writing, 
reading, and mathematics. The 130 tutorial modules are classified into five 
levels of proficiency, from third grade (Level I) to first-year college 
(Level V). In teaching freshman composition, we have found Levels IV 
(eleventh grade) and V (first- year college) to be the most useful. Each of 
the more than 100 courses or modules consists of a brief pretest; a tutorial 
course with sample sentences, explanations, and multiple-choice questions, 
followed by responses to student answers; and a ten-question posttest. 
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In addition to administering individual courses, the CSR system can 
administer diagnostic and prescriptive tests for students in particular areas 
(e.g. punctuation) and prescribe a series of courses based on student 
performance. The system also generates records of all student activity, 
making it easy for instructors to track their students' progress. 

Two Target Areas and Results in Freshman Composition 

We targeted our study to two areas: freshman composition and 
developmental English. Students with lower than an 800 SAT composite 
score and less than a 2.5 GPA are placed in developmental English and 
given an essay placement test to see if they should remain there; likewise, 
students in basic composition are given an essay placement test to see if 
they should move back or forward one level. The University of Tampa is a 
private school with a total enrollment of 2500 full and part-time students, 
and a growing number of foreign and ESL students. 

Funded by a foundation grant, we studied the use of CSR in writing, 
in both freshman composition and developmental classes. What we found 
was both encouraging and discouraging, both promising and puzzhng. 

The results most favorable for the use of CSR came from our 
research into freshman composition, in comparisons of control and 
experimental groups. The experimental group of 17 students had a 
regularly assigned schedule of CSR writing modules from grade level 13 
every Friday, for a total of 24 packets during the spring semester of 1992. 
The control group of 16 students had in-class, teacher-led grammar 
exercises every Friday, for a total of 24 exercises. 



Both groups were given an essay placement test in the first week of 
the semester, a Writing Apprehension Test (Dale-Schall reliability=.92) in 
the second and fourteenth weeks of class, and a final essay exam in the 
fifteenth week. Records indicate that 59% of the control group had taken 
developmental English, while 50% of the experimental group had taken the 
developmental course. 

SAT average scores for the control group and experimental groups 
were 830 and 813, respectively. The control group had a higher grade (B) 
on the placement test than the experimental group (C), but scored slightly 
lower in final grade averages (B-) than their experimental counterparts 
(B). This result suggests that regularly scheduled use of CSR's BASIC 
SKILLS program may help to improve students' overall performance in 
freshman composition. 

Both groups scored 80 on the Writing Apprehension Test (WAP) 
given in week two. The control group gained one point by week fourteen, 
while the experimental group gained two points. The range of possible 
scores on the WAP is 26 (high apprehension) to 130 (low apprehension), 
with an average score of 75. Students in both groups, th n, displayed 
slightly above average confidence in their writing abilities. 

In addition to comparing students in areas such as writing 
apprehension and attendance, we kept records of student errors in essays 
during the semester. In freshman composition, we found that students in 
the experimental group had, on average, fewer sentence fragments, run- 
ons, and comma errors at the end of the semester than the control group. 
Both groups had trouble with possessives and apostrophes tiiroughout tiie 



semester, and we recommended that CSR add grammar modules in these 
a^eas. 

In informal polling, students said they enjoyed CSR and attended the 
electronic sessions on Fridays because of the relaxed, self-paced 
atmosphere. Students were motivated to complete the series of packets and 
especially liked CSR's on-screen help. They were never simply told an 
answer was wrong, but were also given an explanation as to why it was 
wrong. Students found CSR as a writing tutor to be flexible, user-friendly, 
and patient. They freely asked questions of the instructor, and they 
enjoyed the interaction among themselves as they helped one another to 
figure out troublesome grammar situations or recall rules. One student said 
she felt a sense of community among the CSR users. Another said he iiked 
being able to retake the packets or take a tutorial to polish rusty skills. 
Overall, the student response to CSR as a regular part of the freshman 
composition course was favorable: students hked it and recommended its 
continued use for future semesters. 

While the computer-assisted group did somewhat better in final 
grades than the control group, the biggest difference was in attendance. 
Our experimental group missed an average of only 3 days during the 
semester, while the control group missed an average of 5 days. This result 
lends weight to students' statements in favor of CSR. 

In comparisons of placement test scores on a written essay exam, the 
experimental group scored an average of C while the control group's 
average score was B. Interestingly, the group that began with the higher 
placement score of B finished with slightly lower final grades and poorer 
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attendance than the computer-assisted group, but its pass rate was higher 
than the experimental group *s. 

Computer as Tutor in Developmental Composition: 
Some Findings 

According to Brothen (1992), "Computers can help developmental 
students in an important way. Developmental students often need more 
individual attention than traditional methods deliver; the right computer 
software in the right environment can provide this** (p. 32). But what is 
the best way to use computers with developmental writing students? 

In a separate study of two developmental English composition 
classes, we compared two different ways of using CSR's BASIC SKILLS 
program to try to determine which one is more helpful. Specifically, we 
wanted to know whether the program is more beneficial when modules are 
assigned individually on a needs-only basis or when a battery of modules is 
assigned to a whole group of students on a regular schedule. We expected 
that, if the BASIC SKILLS program helps students to write more 
correctly, then students who use the program regularly would probably 
perform better (or at least show more improvement) than those who used it 
only occasionally during the semester. 

To test this theory, we had an experimental group (n=9) meet one 
period per week in the computer lab, where all the students took the same 
CSR modules in grammar, punctuation, spelling, and usage. In all, this 
group completed 22 tutorial modules, The control group (n=20) never met 
as a class in the computer lab. Instead, each student was assigned particular 



CSR modules according to the instructor s perceptions of each student's 
needs. Control group students were to take these modules outside of class 
time, and completion of assigned modules was one criterion in determining 
the course grade. The control group received traditional classroom 
instruction in the areas of grammar, punctuation, and usage. 

To assess the performance of the two groups, we tracked ten 
different kinds of errors (e.g. sentence fragments, subject-verb agreement, 
spelling, etc.) in each of four essays which all students wrote during the 
semester. All ten errors were in areas covered by the CSR modules 
systematically assigned to the experimental group. We expected (and 
hoped) that, in general, the number of errors in both groups would 
decrease as the semester vv^ent on. What we wanted to know was whether 
there would be any significant differences between the two groups in the 
rate of decrease in errors. The results surprised us. While the number of 
errors made by the control group decreased fairly consistently from Essay 
1 to Essay 4, in the experimental group the number of errors actually 
increased steadily from Essay 1 to Essay 3 before dropping off only 
slightly in Essay 4. 

But before we leap to the conclusion that the more developmental 
students work with CSR the more mistakes they make, we need to consider 
some factors other than the number of errors in the essays. First, and 
perhaps most importantly, almost half the students (4 out of 9) in the 
experimental group were ESL students. Secondly, although the 
experimental group's errors increased as the semester progressed, their 
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overall performance for the course was roughly the same as the control 
group's: C average. 

Similarly, although the experimental group's grades on essays 
decreased steadily, they started v^ith a higher average (80) on Essay 1 than 
the control group (77), and the overall decrease was not very great (80 on 
Essay 1 down to 77 on Essay 4). The control group's essay grades also 
declined, but very slightly — from 77 on Essay 1 to 75 on Essay 4, Several 
factors, therefore, complicate the initial impression that the experimental 
group's regular, weekly work on CSR was detrimental. 

Considerations and Conclusions for Developmental Writing 

Before we attempt some tentative conclusions, we should look not 
only at the numbers of errors made by the two developmental writing 
groups but also at the kinds of errors made. As might be expected, the 
experimental group, with its much higher percentage of ESL students, 
made significantly more errors in verb forms , use of commas, and 
capitalization. The large differences here may well be due to the higher 
percentage of ESL students. Two areas in which the experimental group 
showed dramatic and consistent improvement were sentence fragments and 
capitalization errors. In one area, the use of commas, the experimental 
group's performance worsened dramatically: the number of comma errors 
in Essay 4 was more than double the number in Essay 1, despite the fact 
that all students in this group had taken three separate CSR modules on uses 
of the comma. 
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The control group, by comparison, performed significantly worse 
than the experimental group in two areas: spelling (even though this group 
had a much smaller percentage of ESL students) and sentence fragments. 
The most dramatic improvement for the control group came in the areas of 
sentence fragments, verb forms, commas, apostrophes, and hyphens. 
Spelling was the only area in which this group's performance was worse in 
Essay 4 than in Essay 1, but even that area showed substantial improvement 
in Essays 2 and 3 before the regression in Essay 4. 

Considering the differences in the composition of these two 
developmental English groups, especially in number of students and 
percentage of ESL students, we need to be careful in framing and 
qualifying our conclusions. If the experimental group had consisted of the 
same number of students as the control group, or the same percentage of 
ESL students, perhaps the results would be different. However, once we 
grant that possibility, we must admit that this part of our study seems to 
indicate 

that assigning CSR modules to developmental students individually, on a 
needs-only basis, may be more effective than assigning a fixed sequence of 
modules to the entire group. 

How can we interpret this outcome? It may be that student 
motivation to learn and apply a particular rule of grammar or punctuation 
is greater when the instructor has shown the students where they have made 
errors in a particular area, assigning a CSR unit to help remedy the 
problem. The applicability of the CSR exercise to the student's writing 
would, in such cases, be quite obvious. The student would be less likely to 
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regard the CSR exercises as mere busywork. On the other hand, students 
who are assigned CSR courses indiscriminately, regardless of their 
performance on papers, may be less likely to see the connection between 
the computer exercises and their actual writing. The result might be that 
even though they are "learning" principles of correct grammar and 
punctuation, they are failing to apply them to their writing. 

It is difficult to explain why the experimental group's average 
number of errors actually increased - and rather consistently — during the 
coiTse, It may be that for this group a little knowledge was a dangerous 
thing. One student who had used commas as sparingly as if they were 
dollars was suddenly sprinkling them everywhere. Another student, who 
had been putting them everywhere, learned to hoard them like gold. But 
although such a reaction seems to have occurred in the experimental group, 
it did not occur in the control group which, quite to the contrary, showed 
significant improvement in the use of commas. 

The results of this part of our study suggest that in developmental 
writing courses the CSR BASIC SKILLS program may be more effective 
when students are assigned modules according to their individual needs. 
When a whole group of students is assigned a fixed set of modules, the 
program appears to be less effective. 

Summary of the Two Studies 

Taken together, our two studies of the use of CSR's BASIC SKILIS 
in composition classes caution against overgeneralization with regard to the 
effectiveness of this or similar programs. Our study of regular freshman 
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composition classes generally confirms Miami-Dade's finding that 
computer-assisted students perform better, on average, thrji students who 
receive only teacher-delivered instruction. 

Our studies, however, point to a striking difference between regular 
composition groups and developmental writing groups. The regular 
freshman composition students who used the computer program according 
to a fixed schedule of modules clearly profited from it and outperformed 
their noncomputer-assisted counterparts. On the other hand, 
developmental writing students who followed a prescribed series of 
modules did far less well than other developmental students who received 
teacher-delivered instruction in grammar and mechanics and who used the 
computer program on a needs-only basis. It seems that, while the average 
first- year college student is likely to benefit from taking a broad range of 
GSR's computer-delivered writing courses, developmental students should 
focus more narrowly on a few areas of grammar and mechanics in which 
they most need improvement. 

Ultimately, the most important contribution of GSR's BASIC 
SKILLS and similar programs is that they free instructors to concentrate 
on larger and deeper writing issues such as purpose, audience, focus, and 
adequate development of an idea. They enable us to concentrate on helping 
the student write something that is not only correct but is also worth 
wrir^'ng and wortl"! reading. 
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